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ABSTRACT: Researchers have long focused on disaster resilience to mitigate calamity disruption. Disaster 
resilience is a complex and multi-faceted concept that is challenging to measure. Quantitative methods have 
traditionally been used to assess disaster resilience, but a growing interest in qualitative methods like open-ended 
interviews has emerged to understand experiences and perspectives. To gain deep and consistent knowledge, an 
open-ended interview should focus on an interviewee s point of view and ask follow-up questions from a knowledge 
base that consists of relevant information; otherwise, this can lead an open-ended interview to deviate from the 
interviewee s point of view to the interviewer 5 point of view. In contrast to what is desired, individual interviews 
with last year's students in the field of civil engineering with a predefined and limited knowledge base 
demonstrated inconsistency in asking a follow-up question from an already existing open-ended interview. To 
tackle this gap, firstly, we suggest a knowledge base that can be built from peer-reviewed papers published in the 
disaster resilience field; secondly, we suggest a Natural Language Processing based Decision Support System 
using Sentence Embedding that can analyze the interviewee 5 response and find resources from the knowledge base 
to assist the interviewer in making a consistent follow-up question. 


KEYWORDS: Disaster resilience; Decision support systems; Open-ended interviews; Knowledge management; 
NLP 


1. INTRODUCTION 


Disaster resilience is a critical aspect of construction technology that plays a pivotal role in mitigating the impacts 
of various natural and human-induced hazards on built infrastructure (Malalgoda, Amaratunga, & Haigh, 2014). 
In recent years, there has been an increasing emphasis on enhancing disaster resilience in the construction industry 
due to the rising frequency and intensity of disasters worldwide (Harrison & Williams, 2016). Ensuring the 
resilience of constructed facilities not only safeguards public safety but also minimizes economic losses and 
facilitates rapid recovery in the aftermath of disruptive events (Ouyang, Duefias-Osorio, & Min, 2012). 


The challenges posed by disasters necessitate a comprehensive understanding of the factors influencing resilience 
in the context of construction projects. Traditional research methodologies, such as closed-ended interviews and 
surveys, have been instrumental in gathering valuable data on disaster resilience (Cai et al., 2018). However, these 
methods often fall short in capturing the full depth of participants' experiences and viewpoints, leading to potential 
biases in data collection. 


The specific research objectives of this paper are as follows: 


1. To investigate the impact of consistency in open-ended interviews on disaster resilience measurement 
within the disaster resilience domain. 

2. To develop and implement advanced Natural Language Processing (NLP) based Decision Support System 
(DSS) with sentence embedding techniques to enhance data collection in open-ended interviews. 

3. To create a knowledge base that aggregates and organizes peer-reviewed papers and experts' insights 
related to disaster resilience in construction projects. 


The research questions guiding this study are: 


Research Question 1: How does consistency in open-ended interviews influence the reliability and depth of data 
collected for disaster resilience measurement in construction technology? 
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Research Question 2: Can leveraging NLP and sentence embedding techniques enhance the contextual relevance 
of follow-up questions in open-ended interviews within the construction technology domain? 


Research Question 3: How does the proposed decision support system, empowered by the knowledge base, 
improve data collection and analysis in open-ended interviews on disaster resilience in construction projects? 


This paper addresses the significance of consistency in open-ended interviews concerning disaster resilience 
measurement within the domain of construction technology. We recognize the limitations of conventional 
interview techniques and aim to enhance data collection by leveraging advanced NLP and sentence embedding 
techniques. By utilizing a knowledge base of relevant topics in the field of disaster resilience, our proposed 
approach generates contextually relevant follow-up questions that align more closely with the interviewee's point 
of view. 


The contributions of this work are threefold. In this research, first, we demonstrate the existent level of 
inconsistency in disaster resilience measurement domain. Next, we introduce a knowledge base that aggregates 
and organizes peer-reviewed papers and experts' insights in the mentioned domain. This knowledge base empowers 
our decision support system to identify and generate pertinent follow-up questions for interviewees, facilitating a 
more nuanced understanding of their perspectives. Last, we leverage state-of-the-art NLP and sentence embedding 
techniques to ensure the semantic similarity between the interview responses and the knowledge base, enabling a 
more accurate assessment of disaster resilience. 


Using a decision support system is one method of reducing cognitive errors. To help people with complicated 
decision-making tasks, DSSs offer tools and cognitive aids, minimizing reliance on memory and cognitive 
processes alone (Arnott, 2006). DSS assists people in avoiding biases, mistakes, and oversights that may result 
from impaired cognitive function or flawed heuristics by offloading cognitive burden and offering organized 
advice. Such decision support systems can be implemented and used to improve human performance and decision 
outcomes in a variety of domains. 


In the following sections, we detail our methodology, including the data collection process, the implementation of 
sentence embedding and NLP algorithms, and the evaluation of our decision support system. We also present the 
results of our experiments and discuss their implications for the construction technology field. Ultimately, we 
believe that our approach holds great promise in improving the consistency and depth of data collected from open- 
ended interviews, thereby advancing the measurement, and understanding of disaster resilience in construction 
projects. 


2. LITERATURE REVIEW 


As discussed, interviews serve as the primary data gathering method for disaster resilience measurement. Moreover, 
open-ended interviews offer valuable insights into individuals' perspectives; however, the variation in follow-up 
questions among different interviewers can lead to inconsistency and reduced reliability of data gathered. This 
section examines the focus of existing solutions in various domains, particularly in healthcare, where NLP has 
been applied to assist in decision-making processes. Additionally, the lack of existing any DSS that utilizes NLP 
to aid in interview processes within the domain of disaster resilience will be highlighted. 


The literature review section follows a systematic literature review process as described by (Y. Xiao & Watson, 
2019), with step-by-step details presented in Fig. 1. The literature review commenced by defining a set of keywords, 
namely (NLP OR (natural AND language AND processing)) AND (dss OR (decision AND support AND system*)) 
AND interview to cover the scope of our research. Wildcard characters and special terms were employed to identify 
relevant papers. The keywords were used to search in the abstract, keywords and titles of peer-reviewed papers. 
The search yielded 32, 8, 3, and 11 papers from Scopus, IEEE, ScienceDirect, and PubMed databases, resulting in 
a total of 54 papers. By following Fig. 1, the reasoning and numbers of each step is discussed. 
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Identification 


Records identified through 
database searching 


Screening 


Eligibility 


Records after duplicates removed Records excluded with reason (n = 3) 


n=41 
The article was not about natural language 


Records after screening processing (n=1) 
The article does not describe a decision support 


n=38 system (n=1) 


Using social media instead of interview (n=1) 


Papers after full-text eligibility 
checking language processing methods (n=8) 
Review of other related works (n=1) 


The article was not utelising any natural 


Records excluded with reason (n = 11) 


NLP was only discussed as a future work (n=1) 
NLP was only listed without further details(n=1) 


Inclusion 


Studies included 


n=27 Additional records found through forward 
backward searches 


Total number of studies included 
n=36 


Fig. 1: Literature review's strategy applied by following Xiao’s systematic literature review process (Xiao and 


Watson 2019). 


The list of selected papers, along with their domain classifications, is presented in Table 1. The classification was 
done by finding relevant keywords to a specific field that an NLP based DSS was designed for. The classification 
categories included healthcare, engineering, HR, law, and business domains. Healthcare classification was related 
to any health-related papers and engineering ones were the papers mostly focusing on engineering fields like 
mechanical engineering, constructions, and related topics. Any paper within the concept of law, court, and 
advocacy sat within the classification of law leaving HR related ones for hiring related topics and the only business 
one addressing an NLP based DSS within an enterprise. 


Table 1: List of selected papers from systematic literature review. 


Author and Year Title Domain 
(Bazzan, Echeveste, Formoso, An Information Management Model for Addressing Residents’ Complaints Engineering 
Altenbernd, & Barbian, 2023) through Artificial Intelligence Techniques. 
(Afshar et al., 2023) Deployment of Real-time Natural Language Processing and Deep Learning Healthcare 
Clinical Decision Support in the Electronic Health Record: Pipeline 
Implementation for an Opioid Misuse Screener in Hospitalized Adults. 
(Sultanum, Naeem, Brudno, & ChartWalk: Navigating large collections of text notes in electronic health Healthcare 
Chevalier, 2022) records for clinical chart review. 
(Yadav & Sharma, 2023) A novel automated depression detection technique using text transcript. Healthcare 
(Lau, Zhu, & Chan, 2023) Automatic depression severity assessment with deep learning using Healthcare 
parameter-efficient tuning. 
(Huang, Liu, & Lee, 2023) Talent recommendation based on attentive deep neural network and implicit HR 
relationships of resumes. 
(J. Wang et al., 2022) PhenoPad: Building AI enabled note-taking interfaces for patient encounters. Healthcare 
(Chaichulee et al., 2022) Multi-label classification of symptom terms from free-text bilingual adverse Healthcare 
drug reaction reports using natural language processing. 
(Fujimori et al., 2022) Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence- Healthcare 


Based Decision Support Systems in Emergency Departments: Quantitative 
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and Qualitative Evaluation. 


(Barale, 2022) Human-Centered Computing in Legal NLP An Application to Refugee Status Law 
Determination. 
(Rachana, Vishwas, & Priyanka, HR based Chatbot using Deep Neural Network. HR 
2022) 
(C. Wang et al., 2022) A Multi-modal Feature Layer Fusion Model for Assessment of Depression Healthcare 
Based on Attention Mechanisms. 
(Flores, Tlachac, Toto, & Transfer learning for depression screening from follow-up clinical interview Healthcare 
Rundensteiner, 2022b) questions. 
(Flores, Tlachac, Toto, & AudiFace: Multimodal Deep Learning for Depression Screening. Healthcare 
Rundensteiner, 2022a) 
(X. Yang, Joukova, Ayanso, & Social influence-based contrast language analysis framework for clinical Healthcare 
Zihayat, 2022) decision support systems. 
(Jan et al., 2021) The role of machine learning in diagnosing bipolar disorder: Scoping review. Healthcare 
(Jenkins et al., 2021) User testing of a diagnostic decision support system with machine-Assisted Healthcare 
chart review to facilitate clinical genomic diagnosis. 
(Barr et al., 2021) An Audio Personal Health Library of Clinic Visit Recordings for Patients and Healthcare 
Their Caregivers (HealthPAL): User-Centered Design Approach. 
(Toto, Tlachac, & Rundensteiner, Audibert: A deep transfer learning multimodal classification framework for Healthcare 
2021) depression screening. 
(Ivanchikj, Serbout, & Pautasso, From text to visual BPMN process models: Design and evaluation. Business 
2020) 
(Uttarwar, Gambani, Thakkar, & Artificial intelligence based system for preliminary rounds of recruitment HR 
Mulla, 2020) process. 
(Bautista, Aló, & Wang, 2020) Deep Learning, Cloud Computing for Credit/Debit Industry Analysis of Law 
Consumer Behavior. 
(Z. Xiao, Zhou, Chen, Yang, & Chi, IfI hear you correctly: Building and evaluating interview chatbots with active HR 
2020) listening skills. 
(Berquand et al., 2019) Artificial Intelligence for the Early Design Phases of Space Missions. Engineering 
(Mai et al., 2018) Modeling Security and Privacy Requirements: a Use Case-Driven Approach. Law 
(Kramer & Drews, 2017) Checking the lists: A systematic review of electronic checklist use in health Healthcare 
care. 
(Saloun, Ondrejka, Maltik, & Personality disorders identification in written texts. Healthcare 
Zelinka, 2016) 
(Højen, Elberga, & Andersena, SNOMED CT adoption in Denmark-why is it so hard? Healthcare 
2014) 
(Ku & Leroy, 2014) A decision support system: Automated crime report analysis and classification Law 
for e-government. 
(Bagheri, Ensan, & Gasevic, 2012) Decision support for the software product line domain engineering lifecycle. Engineering 
(Huang et al., 2011) Lessons learned in improving the adoption of a real-time NLP decision Healthcare 
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support system. 


(Santelices et al., 2010) Development of a hybrid decision support model for optimal ventricular assist Healthcare 


device weaning. 


(Young et al., 2007) Runtime application of Hybrid-Asbru clinical guidelines. Healthcare 


(Sharda, Das, Cohen, & Patel, 2006) Customizing clinical narratives for the electronic medical record interface Healthcare 


using cognitive methods. 


(Warren, 1998) Better, more cost-effective intake interviews. Healthcare 
(Warren, Warren, & Freedman, Interviewing expertise in primary care medicine: A knowledge-based support Healthcare 
1994) system. 


Fig. 2 illustrates the distribution of covered domains, with healthcare prominently represented. While other 
domains are gaining attention, healthcare remains the dominant focus in NLP-based DSS research. Of particular 
significance, the inclusion of disaster-related keywords in our search strategy consistently yielded zero papers, 
underscoring the absence of NLP-based DSS designed for disaster-related open-ended interviews. Hence, this 
paper addresses the imperative need for such a system and provides a solution to bridge this gap in research. 


Business: 2.8% \ 


Law: 11.1% | 


HR: 11.1% 


Engineering: 8.3% 


Healthcare: 66.7% 


MB Healthcare MM Engineering HR W Law MW Business 


Fig. 2: Percentage of papers using NLP for DSS by domain. 


3. METHOD 


This paper introduces a two-stage design aimed at enhancing disaster resilience open-ended interviews. Initially, 
open-ended interviews were conducted with selected participants using a limited knowledge base. Each participant 
was provided with two open-ended questions along with their respective answers. The participants were then asked 
to generate follow-up questions based on the provided knowledge base. This stage aimed to assess the current level 
of discrepancy in existing open-ended interviews. The second stage presents our designed framework, an assistant 
tool, aimed at enhancing the open-ended interview process. This framework incorporates a modifiable and decent- 
sized knowledge base. Additionally, we propose the utilization of an NLP technique to facilitate the decision- 
making process by offering suggestions to the interviewer. 


Ideally, a follow-up question should align with both the knowledge base and the interviewee’s response, unaffected 
by any other factors. In this scenario, the interviewer's role is that of a mediator between the knowledge base and 
the interviewee. Nonetheless, as highlighted by (Gluyas & Morrison, 2014) “human beings are error prone, and 
the flaws are inherent in human cognitive processes, which are exacerbated by situations in which the individual 


1002 


SECTION D - ENVIRONMENTAL SUSTAINABILITY 


making the error is distracted, stressed or overloaded, or does not have sufficient knowledge to undertake an action 
correctly”. In this paper, this cognitive error is referred to as Interviewer’s Perception. In Fig. 3, which is our 
perception of Jameel’s design for an open-ended interview (Jameel, Shaheen, & Majid, 2018), the interviewer’s 
perception can be seen as the extra factor highlighted in red which should either be eliminated or reduced to certain 
degree for data collection to be more reliable. 


Select a topic to ask 


Start of interview i 
questions from 


Answer the question 


Interviewer ; Interviewee 


Intermewer's percepton 
aa 


Should ask y Ask a follow-up question 
a follow-up from the knowledge base 
Question? relevant to the answer 


Any other 
topic 
questions? 


Knowledge 
base 


End of interview 


Fig. 3: Process of an open-ended interview based on our perception from Jameel’s work (Jameel, Shaheen, & 
Majid, 2018) 


An interview was strategically designed to assess the impact of interviewer’s perception on the process of asking 
follow-up questions during open-ended interviews in the domain of disaster resilience measurement. The interview 
protocol comprised the following steps: 


e Participants: Thirteen students with civil engineering academic backgrounds were recruited for the study. 
Population sample size was determined based on the number of papers published in 2022 with the 
keyword “open AND ended AND disaster* AND resilient*” in Scopus, which yielded thirteen papers. 
The Cochran’s formula for small population sizes was applied with a confidence level of 95% resulting 
in a sample size of 13 (Nanjundeswaraswamy & Divakar, 2021). 

e Interview protocol: Two sets of open-ended questions with answers were developed to elicit rich data of 
decision-making in open-ended interviews of disaster resilience measurement. The interview protocol 
included prompting the participants to elaborate on their responses and give reasoning for their decision- 
making thoughts. Each student is supposed to select two topics for each set of open-ended questions. 

e Knowledge base: A specific knowledge base for the research domain was created by selecting the top 
twelve topics of peer-reviewed papers from Scopus, aligning with the keywords used in the open-ended 
questions in step 2. The topics were directly extracted from the papers. The purpose of the knowledge 
base was to enhance the interview process by narrowing down the choices for follow-up questions and 
reducing the need for students to possess prior knowledge for asking such questions. 


3.1 Analysis 


In our structured interview design, we recognize the practical constraints of interviewers reviewing numerous 
options during the interview process. As a result, we restricted the number of topics to a manageable dozen for 
each question. Each topic is supposed to cover a chain of thoughts from the interviewee’s point of view. However, 
acknowledging that a dozen topics present limitations, it becomes evident that such a limited number may not 
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encompass every potential point of view expressed by interviewees during an open-ended interview. In practice, a 
more substantial number of topics would be necessary to encompass a broader range of existing knowledge and 
adequately cover an interviewee's perspectives. Conversely, with a larger knowledge base, the probability of each 
choice being seen and selected diminishes. 


To calculate the probability, we can use the complement rule. To calculate the probability of selecting at least one 
topic in two chances equals to the complement of not selecting a topic in two chances. If we consider the number 
of topics as n, the probability of not selecting a topic in one chance is (n-1)/n, and the probability of not selecting 
a topic in two chances is (n — 1)/n x (n — 2)/(n — 1). Using the complement rule, the probability of selecting 
at least one topic in two chances can be calculated as (Lefebvre, 2009): 


Probability of a topic selection in two chances (PTS) = (1 — (n — 1)/n x (n — 2)/(n— 1)) x 100 


Simplifying further: 


n n-2 200 
prs =(=-"—) x 100 = — (1) 
n n n 


Considering our interview scenario with only a dozen topics, the rounded value of Probability of Topic Selection 
(PTS) is 16.67. Obviously, the greater the n, the lower the probability of a choice to be selected and with only two 
topics, each of them will have the probability of 100% to be selected. Let us now examine the selection process 
from the knowledge base, which is executed by the interviewer to choose a follow-up topic. 


In the perfect scenario, we would assume that every student only selected a pair of topics and no other topics for 
the follow-up questions. However, in case of reality, which probably differs from the perfect scenario, we will 
consider the most selected choice added to the second most selected as the probable answer and the number of 
times that they were selected as PA and number of times that other choices were made as SC. Thus, we can simply 
calculate the ratio of discrepancy by using the following formula: 


Discrepancy Ratio (DR) = SC/(PA + SC) x 100 (2) 


A lower DR indicates a closer approximation to a perfect interview with minimal errors, approaching a DR score 
of zero. Since we had two sets of questions, we measured them both separately and reported the result with the 
average of them DRs as we put an equal weight on each of the questions. The maximum value of DR can only be 
achieved if each topic for follow-up question topic is selected exactly once or twice. In this case, PA will be equal 
to 6 (3 for the most selected plus 3 for the second most selected choice) and SC will be equal to total number of 
votes which is 26 (13 students and each of which could select 2 topics) minus the rest of the votes which is 20. By 
applying the formula, the result will be 77%. 77% error is a significant value that can impact data collection; thus, 
measuring DR in a real-case scenario is important and furthermore it implies the significance of this study. It 
should be noted that the value of DR can fluctuate between 0% to 77% with the median of 39%. 


In order to comprehensively assess an interview's thoroughness, consistency, and the presence of discrepancies in 
the selection of follow-up question topics, we have devised a novel metric. This simple metric involves the 
multiplication of PTS and DR, with lower values indicating a more valid and reliable interview. We term this 
metric 'Interview's Inconsistency Mark' and it is calculated as follows: 


Interview's Inconsisteny (IIC) = (PTS x DR)/100 (3) 


The reason that we multiply the values is the importance of DR being zero. It means that if the two obvious topics 
will be selected, it doesn’t matter what is the probability of each topic. Considering our designed interview, the 
Interview's Inconsistency (IIC) can vary between zero and 12.84 (approximately 13). An IIC value of 13 indicates 
an interview with highly unreliable data gathering due to inconsistencies in the interviewer's follow-up questioning. 


4. FINDINGS 


In this section, the obtained results from the interview described in the previous section will be reviewed. The 
outcome of this interview provides insights into the practical aspects of conducting open-ended disaster resilience 
measurements by various interviewers. The interview, which simulates a real-case scenario of open-ended disaster 
resilience measurement, can demonstrate how significant inconsistencies can be in real-world, further implying 
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the need of our designed decision support system. Furthermore, an exemplified DR value will be presented to 
facilitate a comprehensive understanding and performance comparison of our devised framework. Additionally, 
our framework will be introduced, aimed at assisting interviewers in formulating more consistent follow-up 
questions during open-ended disaster resilience interviews. 


Prior to delving into the interview results and our framework, an essential concept gleaned from the literature 
review emphasizes the significance of a robust knowledge base during interview processes. In qualitative and 
open-ended interviews, interviewers are more adept at formulating relevant questions when equipped with 
pertinent information and prior knowledge of the subject (Kallio, Pietilä, Johnson, & Kangasniemi, 2016). As the 
interviewee sees the interviewer as educated and well-prepared, it aids in building trust and rapport. A strong 
knowledge base also enables the interviewer to go into complex subjects in more depth, pose probing questions, 
and elicit perceptive responses. This in turn aids in the collection of reliable data during interviews. Therefore, it 
is essential for interviewers to have access to a broad knowledge base which for instance, it is made from literature 
reviews, professional consultations, and in-depth studies to conduct effective and relevant interviews. 


In addition to a solid knowledge base, interviewers must exercise caution when posing follow-up questions to 
avoid making arbitrary assumptions. Follow-up questions serve the purpose of elucidating or further examining 
specific aspects of the interviewee's response. However, by phrasing their follow-up questions based on their own 
beliefs or preconceived notions, interviewers unwittingly introduce bias or influence the interviewee's answers 
(Hunt, 2009). The objectivity and dependability of the interview data may suffer as a result. Interviewers should 
approach follow-up questions with an open and impartial mindset, allowing the interviewee's perspective to guide 
the dialogue and mitigate potential bias. Interviewers can foster a more accurate and thorough grasp of the 
interviewee's experiences and opinions by actively listening, refraining from asking leading questions, and keeping 
conscious of personal biases. Referring to Wreathall, the skill of avoiding cognitive human errors in such decision 
making can be achieved by investing a lot of time and effort and they need constant investment (Wreathall & 
Reason, 1992). 


4.1 Findings from the conducted interview 


In our designed interview, the first question, the first question yielded 11 and 7 selections for the most and second- 
most preferred choices, respectively. This implies a Disaster Resilience (DR) value of 30.77, and with the pre- 
calculated PTS value of 16.67, the Interview’s Inconsistency (IIC) equals 5.13, indicating a moderate level of 
discrepancy within the range of 0 to 13. On the other hand, in our second designed interview, we obtained 12 and 
9 selections for the most and second most selected choices. Upon applying the formulas, we derived an IIC of 1.35 
representing a favorable level of data reliability and reduced inconsistency within the range of 0 to 13. These 
results indicate values falling within the lower half of the normal distribution (between 0 and 13) concerning real- 
world open-ended disaster resilience measurement interviews. However, this does not negate the possibility of 
encountering discrepancies near values such as 5.13, which align with the median value of 6.5. While these 
numbers serve as indicators, they emphasize the need for caution regarding inconsistency, which can potentially 
undermine the validity of the collected data. 


4.2 Our proposed framework 


To address this concern, our designed framework incorporates two essential steps. First, a knowledge base which 
has enough knowledge related to disaster resilience for the moment that an interviewee gives an answer to a 
question and the interviewer needs to ask a follow-up question from it. Second, a decision-making technique for a 
follow-up question selection from the designed knowledge base. 


Historically, an interviewer’s prior knowledge has been primarily regarded as the knowledge base. This has been 
one of the roots causes where we detected the inconsistency of open-ended interviews in disaster resilience 
measurement exists. Therefore, our primary objective in designing the framework was to establish a 
comprehensive knowledge base. We have identified two primary avenues for obtaining information: the existing 
literature on disaster resilience measurement and the expertise of disaster resilience measurement specialists. 
Given the considerable effort required to access and elicit knowledge from diverse experts, we deemed the first 
option more feasible. We opted for Scopus as our literature database, utilizing automation to extract all relevant 
peer-reviewed papers based on specific keywords pertaining to open-ended questions in the field of disaster 
resilience measurement. This automation allows us to expand beyond a limited number of topics to access 
thousands of thoroughly researched papers, ensuring a reliable and extensive knowledge base. During our 
preliminary tests of the automation system, we successfully retrieved a maximum of 1500 peer-reviewed papers 
for each set of question keywords that were given to the students in our conducted interview. To run queries against 
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the knowledge base, we indexed the knowledge base with Anserini’s Information Retrieval library for Python, 
called Pyserini, that has a low latency for information retrieval (P. Yang, Fang, & Lin, 2017). As a result, the PTS 
value from formula 3.1.1 equates to 0.13, implying an extremely low probability of a topic being selected from a 
knowledge base of this magnitude, assuming equal weightage for each topic. Conversely, this approach instils a 
higher confidence level in our knowledge base, encompassing a diverse range of topics that interviewers can 
choose from, aligning more closely with the interviewee's perspective. 


In the context of decision-making techniques and drawing from relevant papers in the literature review section, 
we propose a methodological approach that utilizes Sentence Embedding techniques to generate follow-up 
questions aligned closely with the interviewee's perspective. Sentence Embedding is an NLP problem that deals 
with identifying text that have similarities based on context, meaning and subject etc. based on which classification, 
generation, syntactic parsing etc. of the text can be done (Ryu, Kim, Choi, Yu, & Lee, 2017).Given this definition, 
it becomes evident why this technique captured our interest. Combined as one of the most recent advancements in 
the field of NLP, it considers a sentence as a whole and find similarities, which in our case, sentence embedding 
plays the role of an interpreter in finding similar topics from a knowledge base of relevant topics in the domain of 
disaster resilience. To query the indexed knowledge base, we used T5 doc2query since it has the primary advantage 
of low retrieval latency, keeping an open-ended interview’s follow up question generation to be in real-time 
(Nogueira, Yang, Lin, & Cho, 2019). We utilized cosine similarity to measure semantic similarity between the 
embeddings. The top one percent of highest-ranking topics were made available for the interviewer’s selection. 
With these considerations, assuming a population of fifteen interviewers using our decision support system, and 
the algorithm identifying 1500 topics for a follow-up question, with a selection of two topics from the top one 
percent (15 topics), the resulting DR value could range from zero to 86.67. Although we analyzed 1500 topics, we 
did not observe significant progress in terms of DR. Nonetheless, it is essential to acknowledge that the user can 
modify the one percent value, and 1500 topics represented one of the highest feasible retrieved numbers from 
Scopus. Nevertheless, the IIC value from formula 3.1.3 in this context will vary between zero and 0.11, which 
stands in stark contrast to the actual interview with an IIC of 5.13, and even the lowest recorded case with an IIC 
value of 1.35, as well as the worst-case scenario with an IIC value of 13. The minimal fluctuation achieved 
represents a significant level of consistency for future open-ended interviews in the domain of disaster resilience 
measurement. 


4.3 Conclusion, future works and limitations 


In this article, we demonstrated the significance of consistency of open-ended interviews in the domain of disaster 
resilience measurement. Furthermore, with a methodological approach, we aimed to address the potential 
limitations of open-ended interviews by utilizing sentence embedding techniques and introducing a knowledge 
base to generate contextually relevant follow-up questions. By leveraging sentence embedding techniques and a 
knowledge base generated from peer-reviewed papers, our approach enables interviewers to gather more 
comprehensive and contextually relevant data during open-ended interviews. This enhanced data collection 
process leads to a deeper understanding of participants' experiences and viewpoints, facilitating better-informed 
decision-making for disaster resilience measures in construction projects. 


Moving forward, potential avenues for future research and development include expanding the knowledge base 
by involving experts to review and contribute their insights. Additionally, continuous advancements in Natural 
Language Processing (NLP) algorithms offer opportunities to improve the performance and efficiency of the 
sentence embedding technique used in our system. Further research can explore the integration of additional data 
sources and domains to enhance the decision support system's versatility and applicability in diverse construction 
technology contexts. 
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